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Abstract 

Prevalence of cooperation within groups of selfish individuals is puzzling in that it contradicts with the basic premise 
of natural selection. Favoring players with higher fitness, the latter is key for understanding the challenges faced by 
cooperators when competing with defectors. Evolutionary game theory provides a competent theoretical framework 
for addressing the subtleties of cooperation in such situations, which are known as social dilemmas. Recent advances 
point towards the fact that the evolution of strategies alone may be insufficient to fully exploit the benefits offered 
by cooperative behavior. Indeed, while spatial structure and heterogeneity, for example, have been recognized as 
potent promoters of cooperation, coevolutionary rules can extend the potentials of such entities further, and even more 
importantly, lead to the understanding of their emergence. The introduction of coevolutionary rules to evolutionary 
games implies, that besides the evolution of strategies, another property may simultaneously be subject to evolution as 
well. Coevolutionary rules may affect the interaction network, the reproduction capability of players, their reputation, 
mobility or age. Here we review recent works on evolutionary games incorporating coevolutionary rules, as well as 
give a didactic description of potential pitfalls and misconceptions associated with the subject. In addition, we briefly 
outline directions for future research that we feel are promising, thereby particularly focusing on dynamical effects of 
coevolutionary rules on the evolution of cooperation, which are still widely open to research and thus hold promise of 
exciting new discoveries. 
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1. Introduction 

Cooperation and defection are the two strate gies that 
are us ually at the heart of every social dilemma dDawes , 
1980). While cooperative individuals contribute to the 
collective welfare at a personal cost, defectors choose 
not to. Due to the resulting lower individual fitness 
of cooperators the selection pressure acts in favor of 
the defectors, thus designating the evolution of co- 
operation as a dilemma standing on its ow n. Es- 
tablished by Mayna rd Smith and Pried d 19731). evolu- 
tionary game theory (jMavnard Smithi 1 1 982k IWeibull 
1995b iGintisl l2000t iNowaki l2006ah provides a com- 



petent theoretical framework to address the subtleties 
of cooperation among selfish and unrelated individuals. 
The prisoner's dilemma game in particular, is consid- 
ered a paradigm for tackling the problem of cooper- 
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ation (Axelrod, 11984) . The game promises a defect- 
ing individual the highest fitness if facing a cooper- 
ator. At the same time, the exploited cooperator is 
worse off than a defector playing with another defec- 
tor. According to the fundamental principles of Dar- 
winian selection, cooperation extinction is therefore in- 
evitable. This unadorned scenario is actually realized 
in the well-mixed prison er's dilemma game, where de - 
fectors reign supreme dHofbauer and Sigmundl [1998). 
Relaxing the inevitability of a social downfall consti- 
tuted by the well-mixed priso ner's dilemma is the snow- 
drift or hawk-dove game (IMaynard Smith and Price . 
19731) . where mutual defection is individually less fa- 
vorable than a cooperation-defection pair-up. Accord- 
ingly, the snowdrift game allows for stable coexis- 
tence of cooperators and de f ectors in well-mixed pop 



ulations dTaylor and Jonkerl 119781). Comp leting the 
triplet is the stag-hunt game (Skyrms, 2004), which 
together with the prisoner's dilemma and the snow- 
drift game, forms the standard set of social dilemmas 
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that is frequently explored in th e current liter a ture [se e 



Macv and Flachd d2002h: ISantos et al.1 d2006bl) : 
Szolnoki and Perd d2009d) : iRoca et all d2009ah l. Com- 
pared with the prisoner's dilemma, the stag-hunt game 
offers more support for cooperative individuals in that 
the reward for mutual cooperation is higher than the 
temptation to defect. Still, cooperation in the stag-hunt 
game is compromised by the fact that mutual defection 
is individually more beneficial than bein g an exploited 
coope rator, as recently highlighted by IPacheco et alJ 
d2009h . 

An important realization by the pursuit of cooper- 
ation in the context of social dilemmas was the fact 
that the outcome of evolutionary games in structured 
populations can be very differ ent from the well - mixed 
case. In a pioneering work, iNowak and Mavl d 19921) 
showed that the introduction of spatial structure via 
nearest neighbor interactions enabled the cooperators to 
form clusters on the square lattice and so protect them- 
selves against the exploitation by defectors. Following 
this discovery, the impact of the spatial structure on 
the ev olution of cooperation has been investigated in 

Huberman and Glancej, 
Lindgren and NordahlL 
Purred Il994t iGrirrl 
1995 1 : iKillineback and DoebeliL Tl996l: iNakamara et al 
1997t ISzabo and Toket Il998t iBrauchli et all I l999 | 
Szabo et al., 2000; Tanimoto and Sagara, 2007; 
Alonso- Sanzl 120091: iNewth an d Cornforth, 2009), and 
the subject has since been reviewed comprehensively on 
differ e nt occasions dHauertt 12 002: Doebeli and Hauert, 
ISzabo and Fathi l2007t IRocaetall 12009a). 
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Notably, the theoretical conjecture that spatial struc- 
ture may promote cooperation, or at least sustain a 
multitude of co mpeting strategie s has been confirmed 



experimentally (Ke rr et all 120021) . but there also exist 



evidences that spat ial structure may not ne cessarily 
favor cooperation (Hau ert and Doebeli , |2004 . since 
the impact of the spatial structure on the evolution of 
cooperation depends on the governing social dilemma, 
and due to the difficulties associated with the payoff 



ranki n gs in experimental and fi eld work ( Mi linski et al 
1997 : Turner and Chad. Il999h . it is certainly good 
practice to test new mechanisms aimed at promoting 
cooperation on different evolutionary games. 

The recent shift from evolutionary games on regular 
grids to evolutionary g ames on complex n e twork s 
[for the la tter se e e.g. 
Newmad d2003l) 



Albert and Barabasil ( 2002 ); 
Dorogovtsev and Mendesl (|2003); 
can be considered a step 
Indeed, the shift is 



Boccaletti et all d2006l) l 
towards more realistic conditions 
by no means trivial and bears fascinating results 



recently reviewed by Szabo and Fatd (2007b. Quite 
remar kably, scale-free networks ( Barabasi and Albert! 



1999h turned out to sustain coo peration by all three 



above - described soc i al dilem mas (Sant os and Pacheco , 
2006blch . owing predominantly 



2005: Santos et al 



to the heterogeneity that characterizes their degree 
distribution. Following this seminal discovery, several 
studies have since elaborated on different aspects of 
cooperation on scale-free net works, as for example 
its d ynamical organizat ion dGomez-Gardenes et all 



20071: IPusch et all 120081) . evolution unde r clustering 



dAssenza et al. , 20081). mixing pat terns ( Rong et al 



2007b , memo ry ( Wang et ail l2006h and pa yoff nor- 



maliz a tion ( Santos and PachecoT 20061: Masudal 

all \2 



20071: IWuet all 120071: ISzolnoki et all 2008b). 



well a s its robu st ness i n general dPoncela et all 
2008al) onal 



2007; Chen et al 



attack and error dPerd. l2009b . The body of liter- 
ature devoted to the study of evolutionary games 
on complex network is extensive, aside from the 
scale-free architectur e hostin g the prisoner's dil emma 
(Pacheco and Sant osj 

i Tang et all 120061: lY.-S. Chen and WuL 120071: IPu et al 
F 



2008[ iGomez-Gardenes et all l2008t iFlorfaetal 
2009[ iLi et all 120091: lYang et all l2009bl) and the 
snow dr ift game (IWang et al 1 120061 iLee et all 
20081: IRocaetall l2009bb. cov e ring also small- 



il bnnsl 



world ( Abramson and Kuperman, 
2002; Masuda and Aihar; 



200 it iKimetal 



20031 Tjbm ochi. 



2004 



Santos et al., 2005; Zhong et al., 2006; Tomassini et al 
120061 iFuetall l2007c[ IChen and Waiig[ 12008! 



networks (Holn 
Vukov and Szat 


leetall 2003t Lieberman et al.. 2005 


ol l2005t Wu et all l2006at IChen et al. 


2007 


:lFuet alM 


2007bT]Lufhi et all l2008t lLozano et al. 


2008 


: iLiu et al. 


. 2009: Luthietal. 2009b. Notablv. 



the impact of different interaction topologies has also 
been studied for evolutionary games outside the realm 
of the above-described social dilemmas. Examples 



include the ro ck-paper-sc issors game (ISzabo et al 



as 



2004; Szollosi and Derenvi, 2008), the ultimatum game 
dKuperman and Risau- Gusman, 2008) or the public 
goods game dYang et all l2009d) . and indeed many 
more studies of the latter games on complex network 
are expected in the near future. 

Besides the conditions generated by spatiality 
and complex interaction networks, many different 
mechanisms have been identified that can promote 
or otherwise affect the evolution of cooperation, and 
we mention them here briefly. Aside from network 
reciprocity inherent to games on graphs and complex 
networks, other prominent rules promoting coopera- 



tive behavior are kin selection (Hamilton , I l964albb . 
direct reciprocity dAxelrod and Hamiltorl 1 198 1 
bnr>4 \v>*r-h&r<n gt a i| 



Brandt and Sigmund, 2006; Pacheco et 



_ .20081) 

indirect reciproc ity (Nowa k"and Sigmundl 1998allb| : 
Fehr and Gachterl 120021: iBrandt and Sigmundl 12004 
Nowak and Sigmun dl |2Q05b iTanimotoi l2007cl) and 



so does the environment, and indeed many other factors 
that in turn affect back the outcome of the evolution 
of strategies. Coevolution ary rules can affect the links 
players make (or brake) (Ebel and Bornholdt, 2002b; 
Zimmermann et al., 2004; Zimmermann and Egufluz, 



group selection ( Dugatki n and Mes terton-Gibbons, 

~etall 



1996; iTraulsen and Nowakl bOOeST Traul sen . . 
2008), as recently reviewed in (N owa Ej2006b). 
Moreover, voluntary p articipa tion (IHa uert et al., 



2005: ~[eciui1uz et all 
Santos et all u006a; 



1 20071: iLietaiT 
2007a; Szolnoki et al 



2007; 



120051: IPacheco et all |2006alb : 
Hanaki et all 120071: iBielv et al 
Fu et al 



Tanim otol 12007a 



2002 1 : ISzabo and Hauerl l2002allbl Semmann et al., 
2003 : IHauert and Szabol 120031: [Szabo and Vukovi 
2004; IWu et all 120051: IHauert etail l2007l IChen et all 



2008a; Pacheco et al., 2008 
Perc et all 120081 IChen et all |2008bl IPestelacci et al 
20081: IVan Segbroeck et alT boqtf iFu et al. 



2004; Wu et all 12005b IHauert et all l2007t IChen et all 
2008ch. social d iversity dPerc and SzolnokiL 120081 



2009b] iKun and Scheming! [20091: ISzolnoki and Perc , 
2009 3 ITanimotoi l2009allbUVan Segbroeck et all 12009 ; 
iGraser et al. 



(se e Sectio n[3_JJl, the size of the net 



Santos et al .. 2008), asymme t ric influence of l inks an d 
partner selection dKim et all l2002t IWu et all 120 06b), 
heterogeneous teach i ng acti vity dSzolnoki and Szabol 



work (or po pulation) dRen et al. . 20061 Poncela et al 
120081 i2009h (see Section |3~31). the teaching activity 



2007 : Szolnoki et all l2008cb. an d the impact of long- 
term learning dWang et all 120081) have been suggested 



(or rep roduction capability) (ISzolnoki and Perd 12008 . 
2009 b|) (see S ection 13.31) and mobility of p layers 
(Maieski et 



Vainstein et 



as interesting possibilities that may emerge in real-life 
systems. The necessary ov erlap between i nteractio n 

and replacement g raphs dOhtsuki et all l2007allbl : tion l3.4l i. their age (McNamara et al.. 2008; Stark et al 



Meloni et al 



all [l999l 
all 120071 : 
l2009j 



Vainstein and ArenzonL 2001 



Helbingand YuL 120081 
Drozetall l2009h (see 



2009; 



Wuand Wangl 120071) has also been recognized as an l2008allbl: ISzolnoki et all 120091: lYangetall 12009 



important agonist in the evolution of cooperation. Fur- 
thermore, t he importance of time s c ales in evolu t ionary 



size 



tnermore, me importance ot time scales in evolutionary 
dynamics dPacheco et all l2006bl iRoca et all l2006t 
IPacheco et all 2006a ), t he role of finite populatio n 
dNowak et all 120041: ITraulsen et all 120051 120061) . 



(see Section 13.51. as well as sever al other factors 
(IKirchkampl 1 19991: iGintisl 120031: lAxelrod et aT 



2004 



Hamilton and Taborskv, 2005; Fort, 2008a; 



Hatzopoulos and Jensenl 120081 [Ding et all 
Movano and Sanchez, 2009; Scheming, 
Rankin and Taborskjl 120091: ISzabo et all 
(see Section 13.61 ) that eventually affect the outcome 
of the underlying evolutionary game. Although the 
majority of coevolutionary rules studied so far affects 
the network architecture and size, it is important to dis- 
tinguish these studies from previous, partially closely 
related works whe re networks also change or evolve in 
the course of time dCaldarelli et all ll 998l:IPfeiffer et al. , 
2005; Holme and Ghostal, 2006; iGross and Blasius , 
120081 ICastellano et all 12009b : in particularly so, since 
the term 'coevolution' has in the past been used quite 
frequently and for rather different processes. 

In the continuation of this paper we will review re- 
cent advances on evolutionary games with coevolution- 
ary rules, affecting, as mentioned above, the interaction 
network, the reproduction capability of players, their 
reputation, mobility or age, more thoroughly. Before 
that, however, we give in Section|2]a more technical de- 
scription of the evolutionary games and strategy adop- 
tion rules that we will encounter throughout the paper. 
Following the main body of the review given in Sec- 
tion [3] we conclude our work and give an outlook in 
Section!] 



and the imp act of noise and uncertainties on evolution 



in general dNowak et all 1 19951: [Traulsen et al 



2004 



2009; 



2009; 



2009) 



Szabo et all [20051: IPerd l2006bl: IPerc and Marhll |2006 | 
Per d l2006afd IVukov et all |2006t ITanimotoi 2 007b; 
Percll2007bl:lRen et alll2o'o7HPerc and Szolnokii 120071) 



have been investigated as well. Very recently, ra ndom 
explorations of strategies ( Traulsen et all l2009h and 
simultaneous adoptio ns of different strategies de pend- 



ing on the opponents dWardil and da Silval [2009) have 
also been identified as potent promoters of cooperation. 
Some of these mechanisms will be described more 
accurately in the subsequent sections, but otherwise the 
reader is referred to the original works for details. 

In the focus of this mini review are evolution- 
ary games with coevolu tionary r ules. Initiated by 



Zimmermann et al. d200ll) and by lEbel and Bornholdtl 



(2002b), and in some sense motivated by then very 
vibrant advances in network growth an d evolution 
dStrogatzlEoOlllAlbert and Barabasill2002l) . the subject 
has evolved into a mushrooming avenue of research that 
offers new ways of ensuring cooperation in situations 
constituting a social dilemma. Coevolutionary rules 
constitute a natural upgrade of evolutionary games since 
in reality not only do the strategies evolve in time, but 



2. Evolutionary games 



1 



As noted in the first paragraph of Section[T| the three 
main social dilemmas involving pairwise interactions 
are constituted by the prisoner's dilemma game, the 
snowdrift game and the stag-hunt game. At least one 
of these three games is employed in the majority of the 
works we will review below, and hence we give a more 
accurate description of them in what follows. 

Irrespective of which game applies, players can 
choose either to cooperate or to defe ct. Notably, other 
strateg ies, such as loners [see e.g. lHa uert and Szabol 
d2005h l or punishers [see e.g. iDreber et al.l (120081) 1 are 
also possible, but their inclusion to evolutionary games 
with coevolutionary rules has not yet been considered. 
In general, mutual cooperation yields the reward R, mu- 
tual defection leads to punishment P, and the mixed 
choice gives the cooperator the sucker's payoff S and 
the defector the temptation T. The standard scaled 
parametrization entails designating R = 1 and P — 
as fixed, while the remaining two payoffs can occupy 
-1 < S < 1 and < T < 2. Then, if T > R> P > S 
we have the prisoner's dilemma game, T > R > S > P 
yields the snowdrift game, and R > T > P > S the stag- 
hunt game, as schematically depicted in Fig. Q] With- 
out much loss of generality, this parametrization is of- 
ten further simplified for the prisoner's dilemma game, 
so that T = b is the only free parameter while R — 1 
and P — S = are left constant (thick red line in 
Fig.[TJ. However, since then the condition P > S is not 
strictly fulfilled, this version is traditio nally referred to 
as the weak prisoner's dilemma game (iNowak and May , 
1992). An option is also to use T = b, R = b — c, 
P — and S = —c, thus strictly adhering to the pris- 
oner's dilemma payoff ranking T > R > P > S while 
still having a single tunable parameter in the form of the 
ratio b/c. For the snowdrift game one can, in a simi- 
lar fashion, introduce r e [0, 11 such that T = 1 + r 
and S = 1 — r [see e.g. IWang et aL ( 20061) 1. thereby 
again decreasing the effective dimensionality of the pa- 
rameter space by on e. Note also that r character izes the 
cost-to-benefit ratio dSantos and Pachec ol l2005h and in 
fact constitutes a diagonal in the snowdrift quadrant of 
the T - S parameter plane, as shown in Fig. Q] by the 
dotted blue line. It is worth mentioning that other types 
of para metrization of two -strategy games are possible 
as well dTanimotol 2007a), but we focus on the one pre- 
sented above since it is the most widely used, thus en- 
abling an efficient comparison of different works. 

The most frequently employed setup entails that ini- 
tially each player x is designated either as a coopera- 
tor (s x = C) or defector (s x = D) with equal prob- 
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Figure 1: Schematic presentation of the two-dimensional T—S param- 
eter plane encompassing the stag-hunt (SH), the prisoner's dilemma 
(PD) and the snowdrift (SD) game. Borders between games are de- 
noted by dashed green lines. Dotted blue diagonal depicts the r- 
parametrization of the snowdrift game, while the thick red line shows 
the span of the weak prisoner's dilemma game having T = b as the 
only main parameter. The upper left quadrant represents the so-called 
harmony game (HG). The latter, however, does not constitute a social 
dilemma because there cooperation is always the winning strategy. 



ability, and is placed on one of the nodes of the net- 
work with degree k x . Evolution of the two strategies is 
then performed in accordance with a pairwise compari- 
son rule, during which players accumulate their payoffs 
II A by playing the game with their neighbors. Subse- 
quently, player x tries to enforce its strategy s x on player 
y in accordance with some probability W(s x — > s y ) 
to be specified below. During the simulation proce- 
dure the player x and one of its neighbors y are cho- 
sen randomly, whereby in accordance with the ran- 
dom sequential update each player is selected once 
on average during N (network size) such elementary 
steps, together constituting on e full Monte Carlo step 
dNewman and Barkemal |l999). Alternatively, players 
can be selected sequentially, albeit this may cause artifi- 
cial effects. Independently on whether synchronized or 
the random sequential update is used, however, the time 
evolution is always discrete. The probability of strategy 
adoption W(s x — » s y ) can be defined in several ways. 
If the degree k x of all players is the same and does not 
change in time, the Fermi function 



W(S X -» Sy) 



1 



i + ex P [(n v - n x )/K] 



(i) 



is a v iable option, as proposed by ISzabo and Toke 
dl998l). In Eq ffl K denotes the am plitude of 
~all Ebod: ben et al ' 



noise ( Vu kov et ; 



2007), or equiv- 



alently its inverse (1/K) the so-called intensity of se- 
lection ( Fudenberg et all 20061 : rTraulsen et al . 2007a; 



Altrock and Traulsen, 120091) . In the 7*r — > limit player 
x always succeeds in enforcing its strategy to player y if 
only n v > 11, but never otherwise. For K > 0, however, 
strategies performing worse may also b e adopted ba sed 
on unpredictable variations in payoffs 

JPerEl2006b) 

or 

errors in the decision making, for example. Impor- 
tantly, if the degree distribution of the interaction net- 
work (note that this is a property that may likely change 
due to a coevolutionary rule), at any instance of the 
game, deviates from the case where all players have the 
same degree, the application of the Fermi function may 
introduce additional effects since then the impact of the 
same value of K effectively varies from one player to 
the other. Indeed, if the degree distribution character- 
izing the interaction network is heterogeneous, a more 
successful player {i.e. having a larger payoff) can pass 
its strategy with the probability 



w(s x -» s y ) = (n, - n,)/(A . k g ) 



(2) 



where k q is the largest of the two degrees k x and k y , and 
A = T - S for the prisoner's dilemma game, A = T - P 
for the snowdrift game and A = R — S for the stag-hunt 
game (note that the ranking of payoff elements for each 
specific ga me ensures the positive s ign of Eq.[2]) Intro- 
duced by ISantos and Pachecd d2005l) . it is still a popu- 
lar choice surpassing the difficulties associated with the 
Fermi function described above, albeit with the down- 
side of being unable to adjust the level of uncertainty by 
strategy adoptions. 

Finally, we mention another frequently used strat- 
egy adoption rule in coevolutionary models; namely 
the so-calle d richest-following (or 'learning from the 
best') rule (I Abramson and Kuperman , 2001 : Hu et al, 



20071 IWuetall 120071) . where the focal player al 



ways imitates the strategy of its most successfu l neigh 



bor (j Zimmermann et all 12004c [Equfluz e t all 12005 



iLi et all 120071: iTanimotol E>07a, 2009b). Contrary to 
the preceding two strategy adoption rules, the richest- 
following is completely deterministic, in fact exercis- 
ing the strongest selection between players. Naturally, 
there also exist other microscopic strategy adoption 
rules, such as the win-stay-lose-shift rule where the fo- 
cal player has restricted information on its neighbors, 
for which the rea der is advised to cons ult the compre- 
hensive review by ISzabo and Fath ( 2007) for more de- 
tails. 

We will use the notation introduced above throughout 
this work unless explicitly stated otherwise. Also, any 
deviations with respect to the employed initial setup, 
simulation procedure or the definition of strategy adop- 
tion probability will be noted when applicable. 



3. Coevolutionary rules 

While it is obvious that strategies of players engag- 
ing in evolutionary games evolve in time, the fact that 
other properties characterizing either their individual at- 
tributes or the environment in which the game is staged 
may simultaneously evolve as well gained foothold only 
in recent years. Yet the preceding transitions from well- 
mixed populations to spatial grids and further to com- 
plex networks, and in particular their success in explain- 
ing the evolution of cooperation, are inviting to further 
extensions of the theoretical framework, and indeed, the 
introduction of coevolutionary rules seems like the log- 
ical next step. It should need little persuasion to ac- 
knowledge that links we make with others change in 
time, that all of us age, that our roles in life evolve, 
and that the society we are part of may itself be sub- 
ject to transformations on a global scale. Coevolution- 
ary rules aim to integrate these processes into the frame- 
work of evolutionary games. Perhaps the biggest chal- 
lenge thereby is, how to do this without directly (or ob- 
viously) promoting cooperation. For example, if one 
introduces a rule that, in the course of time, cooperators 
should aim to link only with cooperators and defectors 
only with defectors, it should come as no surprise that 
such a coevolutionary rule will likely favor the evolu- 
tion of cooperation. It is demanding, however, to ex- 
plore and identify successful mechanisms that do not 
attribute special, not to say fictitious, cognitive skill to 
players, and do not use a discriminative set of rules for 
every participating strategy. Thus, coming up with plau- 
sible coevolutionary rules is not straightforward, and 
care must be exercised in order to give both strategies 
equal credentials. Simply because a strategy is bad for 
social welfare it should not be assumed that the individ- 
uals adopting it are less skilful or sly than their oppo- 
nents. In fact, rather the opposite seems to apply. For 
example, defectors should be assumed being just as skil- 
ful by selecting appropriate partners as cooperators. 

In the following we will review coevolutionary rules 
affecting the interactions between players (Section [3~TT i. 
population growth (Section [3~2l ). teaching activity (Sec- 
tion l3.3t . mobility (Section [3~4l i and aging (Section [3~5l l 
of players, as well as related aspects (Section [3~6l l of in- 
dividual and global characteristics that may affect strat- 
egy dominance in evolutionary games. 

3.1. Dynamical interactions 

Coevolutionary rules frequently affect how players 
link with one another and this section reviews examples 
thereof. As we have mentioned above, the result of 
a game with a partner may influence the durability 



of such a connection. In particular, an unsatisfied 
player can easily brake a link to look for a more 
beneficial interaction with another partner. Notably, 
the network itself does thereby not shrink or grow 
in size (for the latter see Section 13 -2b . Instead, our 
aim in this subsection is to explore possible rear- 
rangements of an existing network that is driven by 
the success of players participating in the governing 
evolutionary game. Since coevolutionary rules affect- 
ing t he interactions between pl a yers were proposed 



first ( Zimmermann et al. , 2001; Ebel and Bornholdt, 



2002b), the pertaining literature that has accumulated 



thus far is rather extensive. Works can be partitioned 
into those that emp loyed strategy independent rules 
for li n k adaptations dSzolnoki et all l2008at IPerc et all 
2008; Kun and Scheming, 2009; Tanimoto, 2009a; 



Szolnoki and Per c, 2009c) and those that considered 



strategies or their perfor mances as factors po tentially 
affecting the rewi r ing (Ebel and Bornholdt, 2002b 
Zimmermann et al 1 12004 Zimmermann and Eguflu 
20051; lEaufluz et all 120051 



iPacheco et alTl2006a 



Santo s et all 12006a; iLi et all 20071; iFu et a l., 2007a; 
Tanimotol l2007at iBielv et all l2007t~ l Pacheco et il.. 



2008 



IVan 
iFuet 



Segbroeck et all 120081 Ipestelacci et all 
2008; IFu et all 120081: IChenetall l2008bt iTanimotol 



2009b; F u et all l2009bl IVan Segbroeck et all 120091: 
Oin et all 120091 



Notably, the latter distinction is 
rather crude and sometimes not completely accurate 
since the rewiring can be performed bas ed on a 



secon dary player pr operty, like reputation (IFu et al. 



2008) , attractiveness (|Chen et all I20Q8H) or satisfaction 
(Pest elacci et all 120081) . which are typically related 
with strategy performance over time. It is indeed pos- 
sible to further distinguish the proposed coevolutionary 
rules introducing dynamical interactions to those by 
which the change of the interaction network is driven 
by the urg e to increase the pa y off of t he focal player 



directl y febel and Bornholdtl l2002bl: ISantosetal 
2006al IBielv et all 120071; ILi et all 120071; IChen et al. 



2008 bl IPestelacci et all_ l2008t IVan Segbroeck et al- 
120081 iTanimotol l2009bl: iGraser et all l2009h . and those 
by which the rewiring serves also the increase of the 
payoff but on a global scale, i.e. independently of the 
payoff of t he focal player that is a ffected by t he link 
adaptation (Pacheco et al., 2006a b; Tanimoto, 2007a; 
Fu et all l2007at IPacheco et all 120081: IFu et all l2008i 



2009b; Van Segbroe ck et all 120091) . In the latter 



it is thus not necessary to calculate the players payoff 
prior to rewiring because solely its strategy determines 
the 'life' of a link. 

Summarizing the above, a simplified but useful clas- 
sification of interaction-updating rules is presented in 



Fig. |2 As suggested in the works mentioned last in the 
preceding paragraph, the lifetime of a link may depend 
primarily on the strategies of the players that are con- 
nected with it (type A). From this point of view it is 
straightforward to establish that defector-defector links 
are short-lived if compared to cooperator-cooperator 
links since the former are not beneficial for neither of 
the two involved players, while the later yield mutual 
gains for both. The second set of coevolutionary rules 
evaluates the payoffs originating from the investigated 
link prior to its potential deletion, while the actual re- 
moval takes place only i f a new neighbor may yiel d 
higher benefits (type B) (|Va n Segbroe ck et all [2009). 
And finally, the third set of coevolutionary rules con- 
siders the strategy adoption process as pivotal for de- 
ciding which links to delete and which to keep (type 
C). An example thereof is that the invaded player looses 
all its lin ks except the one with the donor of the new 
strategy (iSzolnoki and Perd l2009alla) . as depicted in 
Fig. I2c). There are several real-life situations that 
can be modeled by the latter rule. From a biological 
viewpoint, the coevolutionary rule can be linked with 
an invasion of the subordinate species and the subse- 
quent replacement by a newborn of the victor. A sim- 
ilar phenomenon can be observed in human societies 
when one changes a job. Typically then the links to for- 
mer coworkers fade and eventually brake, and new ties 
are formed primarily with the coworkers from the new 
working place. Notably, it falls within the same logi- 
cal set of rules if the player that has successfully passed 
its strategy is allowed to increase the number of neigh- 
bo rs that are directly con nected to it, as was proposed 
bv lSzolnoki et al. (2008a). It should not be overlooked, 
however, that the strategy adoption process, triggering 
the deletion and/or addition of links, is itself inherently 
routed in the payoff difference of the considered players. 

An important feature of coevolutionary rules mold- 
ing the interactions among players is also the time 
scale separation between link a nd strat e gy adaptations 
as reported in (ISantos et alj. | 20 06a: Pachec o et al 

Pacheco et al 



2006 



;port 



Szolnoki et al., 
20081 IVan Segbroeck et ail I2008t iFuetal 
Szolnoki and Perc, 2009c; IVan Segbroeck et al. 



2008 



2009; 



Szolnoki and Perd,l2009al) . As the cited works suggest, 



the time scale separation can drastically influence the 
final output of coevolutionary games. This effect will 
also be discussed in the present review. In what follows, 
we will review the coevolutionary rules presented in 
some of these works more accurately. 

In agreeme nt with the actual time-line w e start with 
the work of Ebel and Bornholdtl ( 2002bl) . who pro- 
posed a coevolutionary rule in which a randomly cho- 
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Figure 2: Compai'ative plots of representative coevolutionary rales affecting the interactions between players. In all panels cooperators (defectors) 
are denoted by green (black) circles. Type A [panel (a)]: By 'active linking' the probability to create or to delete a link depends only on its type 
(C - C links are marked by solid, C - D links by dashed, and D-D links by dotted lines). Links to be deleted (created) are marked by red (blue) 
color. Type B [panel (b)]: Adverse ties are deleted depending on the payoffs collected by the players having opposite interests. Subsequently, the 
new link is connected to one of the neighbors of the defeated player. Type C [panel (c)] : Each successful strategy adoption, denoted by a full arrow, 
evokes the deletion of links of the invaded player, except from the one with the 'donor' of the new strategy. 



sen player x is connected to a new neighbor at ran- 
dom. If the new link increases the average payoff of 
the focal player the latter accepts it and disconnects 
from the neighbor it scores worst against. Note that 
this coevolutionary rule indirectly favors the establish- 
ment of cooperator-cooperator links (this pair-up yields 
the highest average payoff) and at the same time facil- 
itates the deletion of defector-defector links. The co- 
evolutionary rule was paired up with strategy mutation 
(lEbel and Bornholdtll2002al) . by which a mutation is ac- 
cepted if it yields a higher payoff for player x than the 
initial strategy [type B; see Fig. Etb)]. Starting from a 
random network with Poissonian degree distribution, it 
was shown that this coevolutionary rule leads to cooper- 
ative Nash equilibria in an iterative prisoner's dilemma 
game with the additional property that no agent can im- 
prove its payoff by changing its neighborhood. Accord- 
ing to the authors, the later may be interpreted as a sort 
of 'ne twork Nash equilibrium' (lEbel and Bornholdt , 
2002bl) . Notably, this coevolutionary rule also affects 
the initial network structure in that the later evolves to 
a statistically stationary state with a broad degree distri- 
bution, suggesting scale-free behavior and giving rise to 
small-world properties, among others. 



Foll owing their precedi n g seminal contributi o n [see 



Zimmermann et alJ (1200 11) 1. IZimmermann et al.1 (2004) 



proposed a coevolutionary rule affecting only defector- 
defector pairs with the motivation that in this pair-up 
both players might be better off if searching for a new 
partner in the context of the prisoner's dilemma game 
[type A; see Fig. Eta)]. It was shown that, starting from 
a random network with a given average degree and the 
richest-following strategy adoption rule, even a small 
probability p of searching for a new partner from a 
defector-defector configuration may substantially pro- 
mote cooperation. Indeed, as low as p — 0.01 were 
shown to uphold practically complete cooperator dom- 
inance across the whole span of the weak prisoner's 
dilemma game (see Fig.fTJ. With respect to the network 
topology, it was reported that the coevolutionary rule fa- 
cilitates the formation of a hierarchical interaction struc- 
ture and may also introduce small-world properties if 
the search for new partners is constrained to the neigh- 
bors of the neighbors. Ho wever, unlike as shown by 



Ebel and Bornh oldt (2002b), the occasional (depending 



on p) break-up of defector-defector pairs has not been 
found leading to broad or even scale-free degree dis- 
tributions. These findings were subsequently extended 



dZimmermann and Eguflu z, 2005; Egufluz et all 12005). 
where it was elaborated on the spontaneous emergence 
of cooperators with extremely high payoffs and the im- 
portant role of this so-called 'leaders' for the global 
sustenance of cooperation. As such, these works can 
be considered as an important prelude to the realiza- 
tion of the fact that scale-free networks constitute an ex- 
tremely favorable environment for the evolution of co- 
operation irrespective of the governing social d ilemma 
dSantos and Pachecoll2005HSantos et alll2006bl) . 

A simple but still plausible coevolutionary rule af- 
fecting links between players has been proposed by 
IPacheco et al. (2006a,b). Exemplifying type A class of 
interaction updating [see Fig.[2a)], players adopting ei- 
ther the strategy C (cooperate) or D (defect) were des- 
ignated a propensity to form new links denoted by c?c 
and do, such that xy links were formed at rates a x a y , 
where x,y e [C,D]. Moreover, each link was assigned 
a specific lifetime depending on the strategy of the two 
connected players given by t xv = y~l , where y xy is the 
corresponding link death rate. With these definitions 
the authors were able to specify mean field equations 
governing the so-called active linking dynamics of the 
network. This coevolutionary rule has been tested on 
the prisoner's dilemma and the snowdrift game subject 
to the Fermi function (see Eq. [TJ gover ning the strat- 
egy ado ption [for additional set-ups see IPacheco et al 



d2006al) l. It was shown that if the time scale associated 
with active linking is much smaller than the one asso- 
ciated with strategy updating the proposed coevolution- 
ary rule leads to an effective rescaling of the governing 
payoff matrix, and thus a shift in the played evolution- 
ary game. For example, the prisoner's dilemma game 
transforms to the coordination game, while the snow- 
drift game transforms to the harmony game [for de- 
tails o n the coordination game see e.g. Sza bo and Fath 
(2007)]. In both cases the cooperation is promoted, in 
turn designating the proposed coevolutionary rule as a 
simple and analytically tractable means of understand- 
ing how selfish and unrelated individuals may be led to 
adopting the cooperative strategy. On the other hand, if 
the ratio between the time scales associated with active 
linking and strategy updating is not small, the interplay 
between these two dynamical processes leads to a pro- 
gressive crossover between the analytic results obtained 
for very fast active linking and the evolutionary dynam- 
ics of strategies taking place on static graphs. The latter 
were found to exhibit different degrees of heterogeneity 
depending on the parameters determining active link- 
ing, yet in general complying well with real social net- 
works having fast decaying tails in their degree distribu- 
tions. Notably, compared to the earlier works reviewed 



ab ove, an important obser vation made in the two papers 
by IPacheco et al. I d2006albh was that the impact of co- 
evolutionary rules may depend significantly on the time 
scales associated with th e strategy and structure (l ink) 
evolution. For example, IZimmermann et alj d2004l) too 
commented on the time scale separation in their model, 
yet the promotion of cooperation was thereby not no- 
tably affected [both slow (p <K 1) and fast (p — » 1) 
rewiring of D - D links was found to be highly effec- 
tive]. Active linking dynamics has also been investi- 
gated in repeated gam es incorporating direct reciprocity 



dPacheco et all 120081) . where additionally the produc- 



tivity of every link connecting two players was evalu- 
ated prior to potential rewi ring. Moreov e r, the active 
linking model proposed by Pacheco et al. (2006b) was 
recently extended bv lVan Segbroeck et alj (120090 to ac- 
count for the impac t of different reactio n s to a dverse 
ties. In particular, Van Seg broeck et al. (2009) addi- 
tionally introduced individual behavioral types of play- 
ers through different values of y, separating those that 
tend to break their links frequently (y close to 1) from 
those that tend to break them rarely (y close to 0). In 
this way both topology and strategy dynamics become 
interrelated. It was shown that populations in which 
individuals are allowed to handle their social contacts 
diversely are more prone to cooperative behavior than 
those in which such diversity is absent. Similarly as in 



Pacheco et all {2006b), it was shown that by an appro- 



priate time scale separation between strategy and net- 
work dynamics the diverse behavioral preferences can 
also introduce a transformation of the governing social 
dilemma, yet so that each individual perceives the same 
game differently. 

Also building on the time scale separation between 
rewiring and strate gy updating is another paper by 
Santos et al. ( 2006al) . where players are able to decide 



which links they want to maintain and which they want 
to change based on local information about their neigh- 
bors [type B; see Fig. [2b)]. A link change is initiated 
if player x is dissatisfied with its connection to player 
y, which is the case if the strategy of player y is to de- 
fect. However, player y also assesses the quality of its 
link to x in the same fashion. If both x and y are sat- 
isfied (which practically means that both are coopera- 
tors) the link between them remains intact. If x wants 
to remove the link and y not (s x = C and s y = D), the 
probability W given by the Fermi function (see Eq. H} 
is invoked. If realized, player x is allowed to redirect 
to a random neighbor of y. If not, x stays linked with 
y. If both x and y are defectors, and thus both want to 
remove the link, then rewiring takes place such that the 
new link keeps attached to x with probability W or to y 



with probability 1 — W. Finally, the authors introduce a 
ratio defined as the time scale associated with the evolu- 
tion of strategies r e (for simplicity equal to one) divided 
by the time scale associated with the rewiring of the net- 
work T a , showing that there exists a critical value for 
this ratio above which cooperators wipe out defectors. 
Moreover, the emerging networks exhibit an overall het- 
erogeneity that is maximal at the critical value and can 
be compared well with diversity associated with realis- 
tic sodjyjietworks ; _The coevolutionary rule proposed 
bv lSantos et al.l ( 2006al) was exte nded by allowing indi 



vidua ls to adjust their social ties dVan Segbroeck etal, 



2008), with the extension that each player was assigned 
an individual willingness < r\ < 1 to rewire unwanted 
social interactions. Accordingly, players with small r\ 
can be considered as loyal to their partners and resilient 
to change, while those with 77 — > 1 are swift in altering 
their links. It was shown that the highest cooperation 
levels can be achieved when the propensity to change 
links is highly strategy-dependent. More precisely, it 
was found very beneficial for the evolution of coopera- 
tion if defectors changed their partners frequently while 
cooperators behaved oppositely, i.e. kept their partners 
for as long as possible. This is indeed expect since de- 
fectors are unable to establish social ties under mutual 
agreement with their partners. On the other hand, co- 
operators are typically much more prone to establishing 
long-term relations and loyalty. Ultimately, these two 
facts lead to the evolution of heterogeneous interactions 
networks where c ooperators are known to pr evail over 
defectors [see e.g. ISantos and Pacheco (200 5D1. 

Related to th e work of ISantos et al.l d2006al) is the 
recent paper by iFu et al. I d2009bl) . the difference being 
that in the latter only cooperators are allowed to switch 
their partners if they act as defectors, and moreover, the 
new partner is sought randomly from the whole pop- 
ulation. It is found that under such a coevolutionary 
rule there exists an op t imal, r ather than critical [com- 



pare with ISantos et al.l (l2006al) l. time scale separation 
between rewiring and strategy updating for which co- 
operation thrives best. Also, the resulting interaction 
topology is different in that the network typically be- 
comes divided into isolated communities of cooperators 
and defectors due to the selective coevolutionary rule 
targeting on ly mixed strategy pairs. Notably, a preced- 
ing study by lFu et aD d2008l) considered partner switch- 
ing also with the aid of reputation, which was defined 
similar to image scoring pro posed a decade earlier by 
Nowak and Sigmund ( 1998b). It was found that coevo- 



lutionary switching of partners based on the reputation 
of nearest and next-nearest neighbors, i.e. preferen- 
tially targeting players with a higher reputation, is sig- 
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Figure 3: Multilevel selection at work. In all panels cooperators (de- 
fectors) are denoted by green (black) circles. A cooperator, strength- 
ened by neighboring cooperators (note that C — C links are beneficial 
for all involved), can pass its strategy to a defector that is weakened 
by neighboring defectors (panel a). Subsequently, the invaded player 
looses its links to other players, except the one with the donor of the 
new strategy (panel b). Due to random link additions, the successful 
invasion of cooperators will repeat itself sooner or later depending on 
t (panel c), ultimately resulting in the disintegration of the defector 
cluster (panel d). Note that this process cannot work in the opposite 
direction, i.e. defectors cannot invade a cluster of cooperators. The 
necessary condition for this mechanism to work is the emergence of 
quasi-homogeneous groups, which occur if strategy adoptions happen 
frequently between new link additions, i.e. if r is large enough. 



nificantly more effective in promoting cooperation than 
seeking a new partner randomly from the whole popula- 
tion. Thus, these r esults underline the importanc e of in- 
direct reciprocity dNowak and Sigmund, 1998a, b|) also 
when individuals can adjust their social ties. 

In addition to the studies reviewed above, similar co- 
evolutionary rules were used to study how scale-fre e 



networks emerge in social systems dLi et all [2007), 
how cooperation in the prisoner's dilemma game can 
be established via the interplay between dynam ical in- 

or in- 



2007a) 



teractions and game d ynamics (Fu et al., _ 
teraction stochasticity dChen et all 2008bl) , how social 
dilemmas in general can t hereb y be resolved dTanimoto , 



2007at IPestelacci et ail l2Q08t iTanimotol l2009bl). as 
well as other soph isticated models faanaki et~ ^2007 



Biely et all 120071) were considered. We refer the 
interested reader to the original works for further 
details, while here we proceed with the review of 
some of the studies that empl oyed strategy indepen- 
dent rules for link adaptations dSzolnoki et all 12 008a: 
Perc et all 2008; K un and Schemin g. 2009: ITanimoto . 



2009atlSzolnoki and Perd.l2009d) . 

Belonging to the third type of interaction-updating 
coevolutiona ry rules Ttype C; see Fig. Etc) 1 is the model 
proposed by ISzolnoki and Perd (120090) . where when- 
ever player x adopts a new strategy all its links, ex- 
cept from the one with the donor of the new strategy, 
are deleted (see Fig. 0, and moreover, all individuals 
are allowed to form a new link with a randomly cho- 
sen player with which they are not yet connected after 
every r full Monte Carlo steps. Note that the random ad- 
ditions of links counteract the deletions following each 
strategy adoption, in turn largely preserving the initially 
random topolo gy and the heterogeneity of the interac- 
tion network dSzolnoki and Perc . 2009a ). It was shown 
that at a sufficiently large time scale separation between 
link deletions and additions, constituted by t, this co- 
evolutionary rules evokes the spontaneous emergence 
of a powerful multilevel selection mechanism, which 
despite the persistent random topology of the evolving 
network, maintains cooperation across a substantial por- 
tion of the T — S parameter plane. Importantly, the pro- 
motion of cooperation is thereby not realized by some 
final outcome o f a coevolutionary rul e, as is for exam- 
ple the case in ( Szolnoki et alll2008al) . but is the conse- 
quence of a dynamical processes that affects the adop- 
tion of strategies on the macroscopic level of evolution- 
ary game dynamics. As Fig.[3] illustrates, the latter man- 
ifests as multilevel selectio n dWilson and Soberl Il998c 
Traulsen and Nowakl 2006 ) that strongly promotes co- 
operation in all major types of social dilemmas. 

Conceptually fitting to the third type of interaction- 
updating coevolutionary rules [type C; see Fig . [2tc)l i s 
also the model introduced by ISzolnoki et al. ( 2008a ). 
where each player x that successfully passes its strat- 
egy (i.e. reproduces in a biological scenario) is allowed 
to form a new link with one randomly selected neighbor 
from its current neighborhood, thereby increasing its de- 
gree k x by one. Thus, successful players are allowed 
to grow compact large neighborhoods that are centered 
around their initial four nearest neighbors. As it is gen- 
erally assumed, the payoff of any given player is accu- 
mulated from all the links with its neighbors. Hence, 
without the normalization by degree, the more links a 
player has the higher its payoff is expected to be. For the 
sake of an easier depiction of player distributions, we 
start from an interaction graph that can be represented 
by a square lattice. Evidently, the additions of new links 
will drive the initial topology away from two dimen- 
sions, yet still allowing us to capture relevant details of 
strategy distributions via a square lattice representation, 
as shown in Fig. H] Notably, the coevolutionary rule is 
independent on whether s x = C or D, and can hence 
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Figure 4: Typical distributions of players on a 100X 100 grid, obtained 
at an optimal {k max = 50; top panel) and a too large (k max = 200, 
bottom pane l) connectivity originat ing from the coevolutionary rule 
proposed by Szolnoki et al. 1 2008a). Full black (open green) boxes 
depict the positions of influential defectors (cooperators) while yellow 
(white) pixels depict the players who are within (out of) their range of 
influence. If the influential players are separated by large disjunct 
territories of influence (bottom panel) the network reciprocity is not 
functioning well. 



be considered as strategy independent. However, since 
the performance of the strategies is clearly definitive for 
who gets to make new links, the rule has at least concep- 
tual similarities with some of the above-reviewed works 
that considered strategies as more directly decisive for 
the outcome of dynamical interactions. Since the co- 
evolutionary rule would eventually result in a fully con- 
nected graph (the latter constitutes well-mixed condi- 



10 



tions), the parameter k max was introduced as the maxi- 
mal degree a player is allowed to obtain. Accordingly, 
the process of making new connections is stopped as 
soon as the degree A; of a single player within the whole 
population reaches k max , whereby this limit prevents the 
formation of a homogeneous system and indeed con- 
stitutes the main parameter affecting the impact of the 
coevolutionary rule. Starting from a square lattice, it 
was shown that intermediate values of k max ~ 50 sub- 
stantially promote cooperation in the weak prisoner's 
dilemma game (see Fig. [TJ governed by Eq. [2] which 
was attributed to the formation of highly heterogeneous 
interactions networks ensuring optimal transfer of in- 
formation between influential players, i.e. those that 
have the highest degree among any other players that 
can adopt the strategy from the influential player via an 
elementary process. The coevolutionary rule was also 
tested against robustness to time scale separation be- 
tween neighborhood growth and strategy adoption via 
the introduction of a parameter q, defining the probabil- 
ity of degree extension after a successful strategy pass. 
Evidently, q — 1 recovers the originally proposed model 
while decreasing values of q result in increasingly sep- 
arated time scales. Although the impact of q was found 
depending somewhat on the temptation to defect b, in 
general values of q > 0.2 yielded insignificantly differ- 
ent results if compared to the q — 1 case. Note that q = 
corresponds to the spatial model without coevolution, 
and hence it is natural that as q — > the promotion of 
cooperation was found fading. The success of interme- 
diate values of k„ mx in promoting cooperation can be ex- 
plained based on the emergence of heterogeneous inter- 
action network s and the disassortative mix i ng of high- 
degree nodes (RongjeTall [20071 l^nimot^, |2009a|). In 
particular, while intermediate values of k max result in a 
highly degree-diverse mixture of p layers, which gener- 
ally pr omotes cooperation [see also lSantos and Pacheco 
(2005)], too large values of k max yield just a few influ- 
ential players with disjunct clouds of homogeneous re- 
gions surrounding them, as shown in the bottom panel 
of Fig. H In the later case, the lack of information ex- 
change between hubs (influential players having large 
degree) defectors can easily survive, thus resulting only 
in moderate cooperation levels. The top panel of Fig. |4] 
on the other hand, features an optimal distribution of in- 
fluential players (i.e. those having large degree), where 
high-degree cooperators can make cooperation prevail 
practically across the whole system. 

It is worth mentioning that the optimal level of co- 
o peration observ e d for a n intermediate value of k max 
Szolnoki et al. (2008a) is conceptually similar to the 




Figure 5: Fraction of cooperators pc as a function of p in the 'con- 
nected influential players' model (Perc et al., 2008). Parameter p de- 
termines the intensity of the information exchange between influential 
players. The fraction of influential players is p = 0.12 and the temp- 
tation to defect is b = 2. The line is just to guide the eye. 



change between influential pl ayers yields the o ptimal 

as is 



2008), 



in 



case when an intermediate strength of information ex- 



environment for cooperation (Per c et al. 
illustrated in Fig. [5] In the later case a fraction p, 
of players that are characterized with a larger teach- 
ing capability are allowed to temporarily link with dis- 
tant opponents of the same kind with probability p, 
thus introducing shortcut connections among the distin- 
guished. These additional temporary connections are 
able to sustain cooperation throughout the whole range 
of the temptation to defect b (see Fig. [TJ. As Fig. 
demonstrates, only minute values of p, constituting a 
moderate intensity of information exchange between in- 
fluential players, warrant the best promotion of cooper- 
ation. 

Strategy-independent coevolutionary rules affecting 
the interactions between players have also been con- 
sidered in the conte xt of distinguishe d players populat- 
ing a square lattice (iPerc et all 120081) . in the context of 
dynamically changing ran dom and scale-free networks 
( Kun and Scheuring , 20091) . as well as in the context 
of random networks with different assortative mixin g 
emerging due to links adaptations dTanimotol [2009a). 
Again, the interested reader is referred to the original 
works for further details, while here we proceed with 
a new section devoted to the review of coevolutionary 
rules introducing network growth. 

3.2. Population growth 

First, it is worth noting that coevolutionary rules 
giving rise to population growth have been consid- 
ered much less frequently than the above-reviewed 
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rules affecting solely how players link with one an- 
other. Indeed, only three works fit into this subsec- 

I ; 1 

tion, the latter being the arXiv contribution by Ren et al. 
( 2006) and two recent papers by IPoncela et al.1 (120081 
120091) . Closely related to the rather general and broad 



equally spaced time intervals tj. The probability that 
any player x, (having payoff II,) in the network receives 
one of the m new links was defined as 



interest in network gr owth dGross andBlasius, 2008: 
Caste llano et all 120091) . the networks formed by the 
players participating in evolutionary games can be sub- 
ject to growth as well, with motivations equivalent to 
those of the broader research field. 

Although never officially published, the work by 



ougn ney ( 

Ren et al.l ( 20061) should be acknowledged as being pi- 
oneering in raising the question how the dynamics of 
an evolutionary game might affect network growth, and 
how in turn the latter affects back the prevalence of the 
competing strategies. For this purpose, the authors pro- 
posed a so-called payoff-based preferential attachment 
rule under the guidance of the r-parameterized snow- 
drift game (see Fig.Q]) and the Fermi strategy adoption 
rule giv en by Eq.pl Inde ed, the preferential attachment 
rule by iRen et al 1 120061) is practically identical to the 
seminal g rowth and preferential attac hment model pro- 
posed by Bara basi and Albert only that in the 
former the probability of linking a new player to an ex- 
isting player x is not determined by its degree k x but 
rather by its accumulated payoff until that time. Not 
surprisingly then (note that in the absence of normaliza- 
tion, similarly as the degree of a player, its payoff will 
typically also increase by one during an update), the co- 
evolutionary rule was found leading to the emergence 
of scale-free interaction networks that are characterized 
by the degree distribution P(k) oc k~ 7 ; the coefficient 
y thereby depending on the scaling of the probability of 
linking a new player to an existing player. In accord ance 
with an earlier study by lSantos and Pachecol d2005l) . the 
emerging scale-free topology due to the coevolutionary 
rule was found highly beneficial for the evolution of co- 
operation in the snowdrift game. Notably, the authors 
also investigated the average path length and the as- 
sortative mixing of the emerging networks, as well as 
the wealth distribution of players. The former two were 
found to be in agreement with observations from realis- 
tic social networks, while the latter was found consistent 
with the Pareto law. 



The work by IPoncela et al.l Q2008) also introduces an 
evolutionary preferential attachment rule that is based 
on the payoffs of existing players, albeit the weak pris- 
oner's dilemma is employed as the governing game and 
the strategy adoption probability is quantified according 
to Eq.|2] More precisely, the network growth starts with 
mo = 3 fully connected players and proceeds by adding 
a new player with m = 2 links to the existing ones at 



Px(t) = 



1 - e + ell x (f) 
2Ul-e + efI v (f)] 



(3) 



where the sum runs over all the players forming the 
network at time t. Moreover, the parameter e e 
[0, 1) controls the weight of the payoffs during the net- 
work growth. For e = all nodes are equiprob- 
able , corresponding to the weak selection limit [se e 
e.g. iTraulsen et al] ( l2007bl) : IWild and Traulsenl d2007h : 
iFu et al .1 d2009al) for recent works related to the latter], 
while for e — > 1 the players with the highest payoffs 
are much more likely to attract the newcomers. The au- 
thors also specified the time interval td for payoff evalu- 
ations and potential strategy adoptions, focusing explic- 
itly on tq/tt > 1 (typically a; 10, although smaller and 
larger values were also commented on), so that accord- 
ingly the network growth was considered to be faster 
than the evolutionary dynamics. It was shown that the 
weak selection limit results in networks having degree 
distributions with exponentially decaying tails, while 
the strong selection limit (e — > 1) yields highly het- 
erogeneous scale-free interaction networks. In agree- 
ment with the ea r lier fin dings obtained on static graphs 
(ISzabo and Fafhl 120071) . it was confirmed that higher 
levels of cooperation are attainable on heterogeneous 
rather than homogeneous topologies, albeit that the dis- 
tribution of strategies with respect to the degree of nodes 
forming the network is different. More precisely, co- 
operators were not found occupying the main hubs as 
on static graphs, but rather the nodes with an inter- 
mediate degree, thus indicating that the interplay be- 
tween the local structure of the network and the hier- 
archical organization of cooperation is guided by the 
competition between the network growth and the evo- 
lutionary dynamics. Notably, similar differences in the 
microscopic organization of the steady state composi- 
tion of strategies were found on static scale-free net- 
works when the payoffs w ere subjected to normaliza- 
tion dSzolnoki et all 2008b). although the discrepancies 
reported by IPoncela et al.l (12008b were solely the conse- 
quence of the c oevolutionary growth process. Indeed, 
in a recent study IPoncela et al. ( 20091) this coevolution- 
ary rule has been studied further to confirm that the 
reported promotion of cooperation hinges not only on 
the final heterogeneity of the resulting network but also 
vitally on the particularities of the growth process it- 
self. In addition, it was shown that under strongly pay- 
off dominated growth conditions so-called super-hubs 
can emerge, which attract most of the links from the 
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other nodes. Although under such conditions coopera- 
tion was found thriving even for high temptations to de- 
fect, it was also noted that the robustness of these find- 
ings may be compromised, o r at least not so stro ng as 



on static scale-free networks (Pon cela et al.L 2007), due 



to the extreme heterogeneity of the star-like structures 
that can be brought about by the coevolutionary network 
growth. 

With the above we conclude the review of coevo- 
lutionary rules affecting the interaction network, ei- 
ther in terms of links players form with one another 
(see Section 13.11 above) or the actual number of play- 
ers participating in the game and the related network 
size. We proceed with the review of coevolutionary 
rules affecting individual properties of players, such as 
their teaching activity (see Section [3~3l >. mobility (see 
Section [3~4T > or age (see Section [331 ). Note, however, 
that some of the above-reviewed coevolutionary rules 
already incorporated and/or affected personal features 
of playe rs, such as for example the loy alty to their 
partners |Van Segbroeck et aL , 2008 , 20091) or influence 
dPerc et all 120081) . albeit always in conjunction with the 
coevolution of the interaction network. In what follows, 
the links and the size of the network are not affected by 
the coevolutionary rules unless explicitly noted other- 
wise. 



3.3. Evolving teaching activity 

Heterogeneity of players has been explored as 
a ben eficial condition for cooperation in several 



forms dWu et alJ.l2006bl:rPerc and Szolnoldl 120081: iFori. 
2008b; Masuda, 2008). It can be easily accepted that 
players are not perfectly identical within a population. 
Some have higher reputation or stronger influence than 
others. These differences can be detected via a bi- 
ased direction of strategy adoptions. More precisely, 
players with higher reputation can spread their strategy 
more easily than if having an average or low reputa- 
tion. In other words, their activity to teach a neigh- 
bor a new strategy is higher. It turned out that one of 
the individual quantities that influences the evolution 
of cooperation most effecti vely is the teaching activity 
( Szolnoki and Szabol 2007 ). Notably, teaching activity 
can also be referred to as th e influence or reproduction 



rate (ISzolnoki et all |2008cJ), with the logical assump 



tion that influential individuals are much more likely 
to reproduce, i.e. have a higher teaching activity, than 
players with low influence. Teaching activity (or the 
synonyms we pointed out) can be introduced into the 
framework of evolutionary game theory via a modified 
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Figure 6: Coev olution of teaching activit y during a strategy adoption, 
as proposed by Szolnoki and Perc (2008). The teaching activity of the 
left player, which is proportional with the size of circle, increases due 
to the successful strategy pass. Note that the right player adopts the 
strategy from the left player, hence the change of color from green 
to black. This strategy-independent (note that the teaching activity of 
the left player increases irrespective of which strategy was passed to 
the right player) coevolutionary rule can result in highly heterogenous 
distributions of teaching activity (see Fig. 0, which were found ben- 
eficial for the evolution of cooperation irrespective of the underlying 
interaction network. 



Fermi strategy adoption rule 



W(s x -> s v ) = w x 



1 



1 + exp[(n > , - U x )/K] 



(4) 



where w x characterizes the strength of influence (or 
teaching activity) of player x. Obviously, w x = 1 for all 
x returns Eq.Q] whereby it is important to acknowledge 
that even if w x < 1 but the same for all x the evolution- 
ary outcome of strategy abundance remains the same, 
only the relaxation times lengthen. Quenched (non- 
evolving) distributions of w x may promote cooperation 
even on homogenous lattice -type interaction topologies 
dSzolnoki and Szabol 120071) . while their application on 
complex networks reveals further that players with large 
teaching activity play a similar role as hubs in highly de- 
gree heterogenous grap hs, such as scale-free networks 



(ISzolnoki et aUl2008d) . We refer the reader to the orig- 



inal works for further details on models using quenched 
distributions of w x , while h ere we proceed w i th the 
review of the two papers by ISzolnoki and Perd (2008, 
2009b) that thus far considered the teaching activity as 
an evolving property of individual players. 

In a social context the strategy adoption can be con- 
sidered as learning from the more successful player. Ac- 
cepting this point of view, it is straightforward to con- 
sider a player who has successfully passed a strategy as 
the one having a higher reputation, and thus a higher 
teaching activity than other players. Implementing this 
idea into a coevolutionary rule, we proposed that when- 
ever player x successfully passes its strategy the influ- 
ence w x increases by a consta nt positive value Aw <s 1 
according to w x — » w x + Aw (ISzolnoki and Peri [2008). 
This coevolutionary rule is illustrated in Fig. [6] It should 
be noted that in this model the term 'reputation' does 
not necessarily have a positive meaning, and thus may 
be in contradiction with the same term used elsewhere 
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dFu et all [2008), where players who cooperated were 
awarded a higher reputation, which expectedly yielded 
higher levels of cooperation. 

Moreover, for the sake of simplicity it was assumed 
that the ev olution of w stops as soon as the highest w x 
reaches 1 dSzolnoki and Perc , 2008). Starting from a 
nonpreferential setup, initially assigning w x = 0.01 to 
every player irrespective of its strategy, it was found that 
there exists an optimal intermediate value of Aw « 0.07 
for which cooperation in the weak prisoner's dilemma 
as well as the r-parameterized snowdrift game (see 
Fig. [TJ is enhanced best. It is in fact understandable 
that only an intermediate value of the increment Aw was 
found warranting the optimal heterogeneity of the dis- 
tribution of w x . Namely, if Aw is small then the values 
of w x simply increase homogeneously for all the play- 
ers, while large values of Aw result in a very quick halt 
of the coevolutionary process, either way resulting in a 
rather homogeneous distribution of the teaching activ- 
ity. Indeed, for both considered evolutionary games it 
was found that using moderate Aw the final distribution 
of w is exponential, in turn attributing the promotion of 
cooperation to the spontaneously emerging highly het- 
erogenous plethora of differently influential players, as 
shown in Fig. [7] It was also shown that the effective- 
ness of the coevolutionary rule increases with the in- 
creasing uncertainty by strategy adoptions K, and that 
the rule is robust to variations of the updating scheme. 
For example, it was verified that an alternative coevo- 
lutionary rule, by which w x was allowed to grow also 
past 1 only that then w x was normalized according to 
w x — * (yv„wx > 1 being the maximal out of all w x 
at any given time) to ensure that the teaching activity 
remained bounded to the unit interval, yielded similar 
results as the halted version. 

A two-fold extens i on of t he above work was made 
in lSzolnoki and Perd d2009bl) . First, the coevolutionary 



rule was no longer considered to be strategy indepen- 
dent. Note that in the preceding work w x — > w x + Aw 
was executed i rrespective of the strategy of player x. 
Conversely, in ISzolnoki and Perd d2009bl) this rule was 
applied separately either only for s x — C (coopera- 
tors) or only for s x = D (defectors). Second, the evo- 
lution of cooperation was examined in all three major 
social dilemma types defined on the T - S parameter 
plane (see Fig. [TJ. It was shown that both versions of 
the coevolutionary rule promote cooperation irrespec- 
tive of the underlying game. Opposite to intuitive rea- 
soning, however, it was revealed that the exclusive co- 
evolutionary promotion of players spreading defection 
is more beneficial for cooperation than the likewise di- 
rect promotion of cooperators. This was attributed to 
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Figure 7: Spontaneously emerging heterogeneous distribution of the 
teaching activity P(w x ) as a result of the coevolutionary rule intro- 
duced by Szolnoki and Perc (2008). Results were obtained for the 
weak prisoner's dilemma game staged on a square lattice. Parameter 
values were: b = 1.05, K = 0.1 and An' = 0.07. 



the fact that the coevolutionary promotion of defectors 
results in a larger fraction of players that are at least 
once affected by the coevolution, ultimately leading to a 
stronger segregation of the population into active (those 
having w x > 0.01; note that the latter is the initial teach- 
ing activity assigned to all) and virtually (or compa- 
rably) inactive (those having w x = 0.01) players than 
the coevolutionary rule affecting cooperators. Accord- 
ing to previous findings on the impact of static distri- 
butions of heterogeneity (IPerc and Szolnokii 120081) . the 
stronger expressed segregation was found directly re- 
sponsible for the better promotion of cooperation when 
defectors rather than cooperators were subjected to co- 
evolution. 

As we have already mentioned (see e.g. Section lXTl i. 
the time scale separation of coevolutionary processes 
may decisively affect the final output of such models. 
This was observed for the coevolution of teaching ac- 
tivity as well. More precisely, the time scale separa- 
tion between the coevolution of teaching activity and 
strategy adoption can be tuned via the introduction of 
a parameter q, defining the probability of increasing w x 
after a successful strategy pass. Evidently, q — 1 recov- 
ers the two originally proposed models while decreasing 
values of q result in increasingly separated time scales. 
Although the impact of q was found depending some- 
what on the type of the considered coevolutionary rule, 
in general, values of q > 0.3 yielded insignificantly dif- 
ferent results if compared to the q = 1 case, thus indicat- 



ing that the findi ngs are robust to this typ e of alterations 
[note that, as in ISzolnoki et alJ (12008a), q — corre- 
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sponds to the model without coevolution, and hence it is 
natural that as q — » the promotion of cooperation was 
found fading]. More precisely, however, since the frac- 
tion of cooperators was found increasing rather steadily 
with increasing values of q, especially for the revo- 
lutionary rule affecting defectors, it is optimal to keep 
the coevolutionary process affecting the teaching activ- 
ity of players paced similarly fast as the main evolution 
of strategies, i.e. q — > 1. 

We thus emphasize, that the above-reviewed coevo- 
lutionary models affecting the teaching activity have re- 
vealed that a simple 'successful become more success- 
ful' principle can result in a heterogenous hierarchy of 
individual properties of players, such that optimal con- 
ditions for the evolution of cooperation are warranted. A 
similarly positive impact of heterogeneity on the spread 
of the cooperative strategy was a lso detected on het- 
eroge nous interaction networks (ISantos and Pacheco , 
2005), hence conceptually linking these two seemingly 
disjoint promoters of cooperation. 

3.4. Mobility of players 



not relying on any type of explicit, genotypic or pheno- 
typic assortment, and also being strategy-independent. 
Due to this minimalist set-up the study provided rather 
general insights into possible effects of mobility. It was 
shown that mobility may indeed promote cooperation 
since it increases the ability of cooperator clusters to in- 
vade and overtake isolated defectors. On the other hand, 
mobility may also allow defectors to escape retaliation 
from a former partner and lead to stronger mixing in a 
population due to increasing interaction ranges of play- 
ers, both of which are known to damp the evolutionary 
success of cooperators. Thus, the impact of m obility 
in the form introduced by IVainstein et al.l (120071) is not 
clear cut. As noted by the authors, further work on this 
is in progress. Importantly, it was also emphasized that 
mobility may be subject to more deliberate coevolution- 
ary rules, taking into account personal preferences of 
players, their strategies, as well as aims. 

An example of t he latter was studied by 



Helbi ng and Yul (120091) . who introduced success- 
driven migration as a possible mediator leading towards 
cooperation in populations of selfish and unrelated 
It was acknowledged already bv lMaieski et al. (1999) individuals even under noisy condition s. In particular. 



that players finding themselves in an unprofitable or un- 
desirable situation frequently choose moving in order to 
free themselves from the negative consequences of that 
situation. Accordingly, mobility can be considered as 
being a coevolutionary process in the sense of strategy 
and/or position alterations that ultimately determine the 
environment of players. Although we were unable to lo- 
cate coevolutionary terminology associated with mobil- 
ity, we review here advances on this topic made during 
the last decade, and indeed consider movements of play- 
ers during the evolution of strategies as being guided by 
rules of coevolution. 

The impact of diffusion on the outcome of a spa- 
tial prisoner's dilemma game via empty sites was ques- 
tioned first by IVainstein and Arenzonl d200ll) . Therein, 
weak quenched disorder introduced in the form of 
empty sites on a square lattice was found beneficial for 
cooperation in the prisoner's dilemma game subject to 
the richest -following strategy update rule. In a follow- 
up paper ( Vai nstein et al. , 2007) the approach was ex- 
tended to allow diffusion of players to nearest-neighbor 
empty sites with a certain probability. In particular, 
two ways of implementing the mobility were consid- 
ered. First, each player was allowed to make an attempt 
at moving only after payoff accumulation and potential 
strategy adaptations were executed in parallel, or sec- 
ond, the moving was attempted prior to the evolution of 
strategies. Importantly, the moving of players was con- 
sidered to be Brownian random walk like, i.e. diffusive, 



succes s-driven migration [see also Hel bing and Yu 
(2008)] was implemented so that, before the strategy 
adoption, player x was allowed to explore potential 
payoffs that it would receive if occupying one of the 
empty sites in the migration neighborhood. The latter 
was typically restrained to nearest and next-nearest 
neighbors of player x. If the potential payoff was 
found to be higher than in the current location, player 
x moved to the site offering the highest payoff and, in 
case of several sites with the same payoff, to the closest 
one. On the other hand, if the current location offered 
the highest payoff among all the empty sites within 
the migration neighborhood, player x did not move. 
It was found that this fairly simple and very plausible 
migration rule promotes cooperation in the prisoner's 
dilemma game on a square lattice (with a fraction of 
empty sites to accommodate moving) irrespective of 
the noise introduced to the system. In fact, three types 
of noise were considered to attest to the robustness of 
cooperation facilitation due to the introduced mobility 
of players. First was the introduction of mutation with 
probability q, second was the introduction of random 
movements not considering the expected success (pay- 
off) with probability r, while third was the combination 
of the two. Additionally, different update rules, adding 
birth and death processes, as well as introducing a small 
fraction of individuals defecting unconditionally were 
considered as well. Irrespective of all these factors 
cooperation was found prevailing in a large region of 
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the parameter space defining the prisoner's dilemma 
game if only the players were allowed to execute 
success-driven migration. 

Recently, mobility is getting increasing attention as 
a means to promote cooperation in social dilemmas 
dDroz et al.l, l2009tlMeloni et al.[|2009h . In the model of 
Droz et al. ( 20091) two types of players are introduced, 
and a random walk of the influential individuals is pos- 
sible irrespective of their strategies. The mobility of in- 
fluential cooperators can have two positive impacts on 
the evolution of cooperation. First, they can spread the 
cooperative strategy among the non-influential players 
having a lower teaching activity, and second, when two 
influential players with opposite strategies meet, the co- 
operator can prevail and thus ensure an effective infor- 
mation exchange between cooperating hubs. Note that 
the importance of the latter has as already been empha- 
sized in Section [XTl (see e.g. Fig.|5]and the pertaining 
text). As expected based on preceding works consider- 
ing mobility as a coevolutionary process, the final out- 
come of the competition between mobile, and thus in- 
fluential, cooperators and defectors is highly sensitive 
to changes in the speed of moving. When the latter is 
too high the influential cooperators cannot benefit from 
their cooperative neighborhoods because they abandon 
them too soon. Similarly, influential defectors eschew 
the negative feedback effect originating from defecting 
neighbors (note that D-D links are nonprofitable for 
both players) because they leave them too fast as well. 
Indeed, high moving speeds generate conditions mim- 
icking the well-mixed regime which is damning for co- 
operators. Thus, only moderate mobility of influential 
players has been found effectively supporting the evolu- 
tion of cooperation. We refer the reader to the original 
works for further details, noting that mobility seems a 
promising avenue of research for future explorations of 
coevolutionary rules. 

3.5. Aging of players 

As the last coevolutionary process we consider ag- 
ing. Indeed, aging is always present, tailoring our in- 
teractions with others and postulating a finite lifespan 
during which we are able to exercise them. It thus 
seems natural to consider aging as an integral part of ev- 
ery evolutionary development, and certainly evolution- 
ary games constitute a prime example thereof. Nonethe- 
less, studies taking aging expli citly into account within 



this realm of research are few. McNa mara et alj (2008) 



recently noted that lifespan might play an important role 
in the evolution of cooperation, albeit their study fo- 
cused on the coevolution of choosiness (see also Sec- 
tion 13.6b rather than age. Moreover, concepts similar 
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Figure 8: Dynamical explanation of cooperation promotion emerging 
due to aging, as proposed by Szolnoki et al. ( 2009). When an old de- 
fector, with a high strategy transfer capability, is imitated by one of 
the neighbors, further spreading of defection is blocked because the 
newborn defector has no chance to pass strategy D further. The new- 
born is not supported by its ancestor (the D-D link is detrimental for 
both), and hence a neighboring cooperator with high age can conquer 
the site of the newborn defector. This procedure occurs repeatedly, ul- 
timately resulting in a practically blocked (more precisely an oscillat- 
ing) front between C and D regions (left panel). Importantly, a similar 
blocking is not present around old (and thus influential) cooperators 
because their cooperator-cooperator links help newborn cooperators 
to achieve higher age, in turn supporting the overall maintenance of 
cooperative behavior (right panel). In both panels the size of players is 
proportional to their age (i.e. their teaching activity). Dashed (solid) 
arrows denote attempted (successful) strategy adoption processes. 



to aging were rece ntly introduced in the voter model 



dStark et all l2008albT) . showing that age and memory 



dependent transition rates can have a positive effect on 
consensus formation. A recent work f ocusing on ag- 



ing wi thin evolutionary games is due to Szolno ki et al 



(2009), and in the following we present a summary of 
the proposed coevolutionary rules for aging as well as 
their main implications. 

Since age is often associated with knowledge and 
wisdom an individual is able to accumulate over the 
years, it was introduced through a simple tunable func- 
tion that maps age to teaching activity (see Section [3~3l 
of the corresponding player. More precisely, w x in Eq.|4] 
was related to the integer age e x = 0,1,..., e max in 
accordance with the function w x = (e x /e max ) a , where 
emax = 99, denoting the maximal possible age of a 
player, serves the bounding of w x to the unit inter- 
val, and a determines the level of heterogeneity in the 
e x — * w x mapping. Evidently, a — corresponds to 
the classical (homogeneous) spatial model with w x = 1 
characterizing all players, a = 1 ensures that w x and 
e x have the same distribution, whereas values of a > 2 
impose a power law distribution of strategy transfer ca- 
pability. Although different age distributions of players 
were considered also as quenched system states, the fo- 
cus was on the study of aging as a coevolutionary pro- 
cess, entailing death and newborns. Two rules were con- 
sidered separately, both starting with e x being assigned 
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randomly from a uniform distribution within the interval 
[0, e max ] to all players, and subsequently increasing all 
e x by 1 after each full Monte Carlo step. Also inherent 
to both was that e x was set to zero for all players whose 
age exceeded e max (effectively this means that a new- 
born follows the dead player). The difference was in the 
way age of players that have just adopted a new strategy 
from one of their neighbors was handled. In the coevo- 
lutionary model A their age was left unchanged, while 
in the coevolutionary model B they were considered as 
newborns, i.e. as soon as player x adopted a new strat- 
egy its age was set to e x = 0. Notably, rules A and B can 
be interpreted rather differently. From a purely biolog- 
ical viewpoint the more successful player replaces the 
neighbor with its own offspring, who therefore initially 
has a limited strategy transfer capability, which corre- 
sponds to rule B. On the other hand, especially in social 
systems, strategy adoptions may not necessarily involve 
death and newborns, but may indicate solely a change 
of heart, preference, or way of thinking, whereby this 
situation corresponds to rule A. Nevertheless, newborns 
in a social context can be considered those that changed 
their strategy recently, and therefore have a low reputa- 
tion initially. Interestingly, it was found that the small 
difference between coevolutionary rules A and B may 
have significant consequences for the evolution of co- 
operation. Foremost, it was found that rule B promotes 
cooperation remarkably better than rule A. However, the 
difference could not be explained by the resulting het- 
erogeneity of the distributions of W r, for example via 
a simi lar reasoning as introduced by IPerc and Szolnokil 
d2008l) . since both rules return power law distributed 
values with rather similar slopes (-0.5 for A and -0.7 
for B). In fact, it was shown that the coevolutionary 
rule B introduces a new powerful mechanism for pro- 
motion of cooperation acting solely on a microscopic 
player-to-player basis, and as such is thus virtually not 
detectable by statistical methods assessing the hetero- 
geneity of the system. The mechanism was found re- 
lying on a highly selective promotion of cooperator- 
cooperator and defector-defector pairs, which hinders 
influential defectors (those having e x close to e max ) to 
spread their strategy effectively across the spatial grid. 
In particular, rule B always leads to influential players 
being surrounded by newborns. Thereby it is impor- 
tant to note that whenever an old defector, with a high 
strategy transfer capability w x , is imitated by one of the 
neighbors, further spreading of defection is blocked be- 
cause the newborn defector has no chance to pass strat- 
egy D further. At that time a neighboring cooperator 
with high age can strike back and conquer the site of the 
newborn defector. As a result the whole procedure starts 




Figure 9: Snapshot of a typical distribution of players on a 100 X 100 
square lattice, obtained by considering players who have adopted a 
new s trategy as newborns [coevolutionary rule B in ISzolnoki et all 
(2009)]. Full black (open green) boxes depict influential defectors 
(cooperators) while yellow and white are all other non-influential 
defectors and cooperators, respectively. The snapshot demonstrates 
clearly that the propagation of defectors is blocked in space as a con- 
sequence of age-related teaching activity. 



again, which ultimately results in a practically blocked 
(more precisely an oscillating) front between C and D 
regions. Crucially, a similar blocking mechanism is not 
present around old (and thus influential) cooperators be- 
cause their cooperator-cooperator links help newborn 
cooperators to achieve higher age, in turn supporting the 
overall maintenance of cooperative behavior. The main 
differences in the propagation of different strategy pair- 
ups are summarized in Fig. [8] while an example of the 
resulting spatial distribution of players is presented in 
Fig. [9] In the latter a player is considered as influential 
if its age exceeds that of any of its neighbors by at least 
e max l2 (qualitatively similar snapshots can be obtained 
by choosing different thresholds as well). 

In model A the situation is significantly different 
since cooperative domains, created around old players 
with high w x , cannot prevail long. Namely, the central 
cooperator who built up the cooperative domain eventu- 
ally dies, and the arriving newborn with an accordingly 
low strategy transfer capability simply cannot maintain 
this domain further, thus giving defectors an opportu- 
nity to win it over. As a consequence of the dynamical 
origin of the observed cooperation-promoting mecha- 
nism brought about by rule B, it is expected that it will 
work in other cases too, for example when the interac- 
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tion graph is characterized by a different topology, by 
other evolutionary games, or by separated time scales 
between a ging and strategy ad aptations, as was shown 
already by lSzolnoki et al.1 J2009I) . 

Finally, it is important to note that the observed 
cooperation-promoting mechanism relying on a dynam- 
ical process is robust even if non-monotonous mappings 
between e x and w x are considered (the oldest individu- 
als may not necessarily be the most influential). Indeed, 
the promotion of cooperation remains intact as long as 
the plausible assumption that very young players should 
have none or very little influence is adhered to. 

Similarly as mobility reviewed in Section 13.41 we 
note that aging as a coevolutionary process seems very 
liable to further studies as well, and we hope this brief 
summary succeeded in wetting the appetite for them. 

3.6. Related approaches 

Aside from thus fa r reviewed c oevolutionary rules , 
there exist ex a mples dKirchkampj fl 999: Gintis, 2003; 
Axelrodetal., 2004; Hamilton and Taborsky, 2005; 



Fort . | 2008ak Hatzopoulos and Jens em 12008! iDinget all 
20091: iMovano and Sanchez! 120091: Scheming! 120091 
Rankin and Taborsky! l2009t ISzaboetall 120091) we 
were unable to classify into the above subsections. 
Without going into much details as it exceeds the scope 
of this mini review, we briefly describe some of these 
related approaches, but refer the reader to the original 
w orks for further de tails. 

Kirchkam3 dl999h . for example, studied the si- 
multaneous evolution of learning rules and strategies, 
whereby the former were determined endogenously 
based on the success of strategies observed in the neigh- 
borhood of any given player. It was shown that en- 
dogenous learning rules put more weight on the proper 
understanding of each player's own experience rather 
than on the experience of an observed neighbor. Co- 
ev olving learning rules were recently considered also 
by lMoyano and Sanchez! (120091) . showing that imitation 
is frequently displaced by replication, in turn leading 
to a rapid decrease of cooperation in the spatial pris- 
oner's dilemma game. On the other hand, imitation was 
found to be superior to global but stochastic imitation, 
thereby facilitating cooperative behavior. The coevo- 
lutionary sej^ctionof strate gy adoption rules was con- 
sider by ISzabo et al. (2009) as well, where the uncer- 
tainty in the Fermi function (see Eq. [T]) was subject to 
evolution as a player-specific property. In particular, in- 
stead of a single K value authors introduced different 
Ki values where i e (1,2 . . . ,n), which were then as- 
signed randomly to the players. As we have already 




Figure 10: Time dependencies of the fraction of initial Kj values (vjf,) 
demonstrate the selection of the most appropriate strategy adoption 
uncertainty K* (promoting cooperation best), as indicated by the red 
solid line. This directly implies the spontaneous selection of an opti- 
mal strategy adoption rule within the scope of the Fermi function (see 
Eq.fTJ, as reported by Szabo et al. 12009). Dashed blue lines depicted 
the extinction of the other K, values. Initially, n = 20 different K{ 
values were distributed on the square lattice of size N = 1000 2 . The 
temptation to defect was b = 1.05. 



noted in Section [2] the uncertainty by strategy adop- 
tions can originate from different sources, ranging from 
unpredictabl e variations in payoffs to errors in the deci 



sion m aking ( Vukov et al. , 2006: Perc!l2006b!lDu et al 



l2009bUWu and Holmd. 120091: iDu et alll2009al) . The pa- 
rameter K, however, can also be considered as charac- 
terizing the willingness of a player to risk a payoff quan- 
tity during a strategy change. Therefore, by using dif- 
ferent values of K, not only the better strategy but also 
the way of strategy adoption can be the subject of an 
imitation process. 

Accordingly, aside from the fact that player y could 
adopt the strategy of player x according to Eq.[T](s A — > 
s y ), an additional independent trail with the same prob- 
ability was made also for the adoption of the imita- 
tion rule (K x — > K y ). It was shown that, if the sys- 
tem is seeded by random initial conditions, the proposed 
coevolutionary rule drives the system towards a state 
where a single K* value (only one of the initial Ki val- 
ues) prevails. This final strategy adoption uncertainty 
is closely related with the parameter value warranting 
the highest cooperation level if a given value of K is 
used for all players at a certain value of the temptation 
to defect b (see Fig. Q}. Naturally, the prevailing K* 
value thus depends also on the topology of the inter- 
action network. The selection process is illustrated in 
Fig-Oil where n — 20 different Ki values were initially 
assigned to the players x e (1,2. . .,N). Summarizing 
the main observation, it was shown that a Darwinian 
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selection rule affecting a model parameter can sponta- 
neously lead to the prevalence of the value that ensures 
an optimal level of cooperation in the system. For fur- 
ther details we ref er the reader to the original work of 
Szabo etalJd2009l) . 

The evolution of altruistic behavior under coevolu- 
tionary rules was studied also in what can be con- 
sidered more expli citly biologic a lly or even humanly 
motivated setting s tGintis, 2003; Axelrodetal., 2004; 
McNamara et all 120081: IScheuringl 120091: bing et all 
2009b . For example, internal norms, being a pattern 
of behavior enforced in part by internal sanctions, such 
as shame, guilt and loss of self-esteem, were found to 
provide support for the evolution of altruistic norms, 
and moreover, via a gene-culture coevolution argument 
an explanation was provided as to why individually 
fitness-reducing internal norms ar e likely to be proso- 
cial rather than socially harmful dGintis , 2003 ). Al- 
though mentioned alrea dy in Section 13.51 the study of 



McNamara et al 



(120081) in fact focuses on the coevolu- 
tion of choosiness, the later relying on cooperativeness 
being used by other individuals as a choice criterion. 
In such a setting competition to be more generous than 
others can emerge, and in this case the evolution of co- 
operation between unrelated individuals can be driven 
by a positive feedback between increasing levels of co- 
operativeness and choosiness. It was shown that, in situ- 
ations where individuals have the opportunity to engage 
in repeated pairwise interactions, the evolution of coop- 
eration depends critically on the amount of behavioral 
variation that is being maintained in the population by 
processes such as mutation. 

Fina lly, we note that lHatzopoulos and Jensen 
(2008) investigated the evolution of cooperation 
in a so-called nongrowth dynamic network model 
with a d eath-birth dynamics ba s ed on tournament 
selection. lHamilton and Taborskvl d2005l) as well as 
Rankin and Taborskvl d2009l) studied the coevolution 
of group structure rather than graph structure in the 
context of generalized reciprocity, while iFortl d2008ah 
considered evolving heterogeneous games as means to 
sustain cooperation. Interestingly, in the later study 
the players had individual payoff elements assigned to 
them for calculating their final payoffs. Accordingly, 
within the realm of the proposed coevolutionary rule a 
player could adopt not only the strategy of the neighbor 
but also its individual payoff matrix elements. It was 
found that if starting with a random heterogeneous 
distribution of payoffs, eventually only a small number 
of definite payoff matrices remained while the oth- 
ers went 'extinct'. On the other hand, if the initial 
rank of individual payoff elements agreed with those 



constituting social dilemma games, the prevailing one 
was found to be the stag-hunt game. Considering the 
latter result in the light of finding s reported within the 
evolving adoption rules model by ISzabo et al.1 (120091) . it 
is possible to raise the question if coevolutionary rules 
as a selection mechanism can spontaneously drive the 
system into a state where mutual cooperation ensures 
the maximal average payoff. Indeed, further studies 
are necessary to clarify this issue. In sum, there are 
few boundaries to imagination when considering what 
coevolutionary rules might affect, and certainly, it 
seems like all facets of existence can be brought into 
consideration. 



4. Conclusions and outlook 

As we hope the above mini review on coevolutionary 
games clearly shows, coevolution is certainly a promis- 
ing concept to follow, as it constitutes the most natural 
upgrade of evolutionary games in the sense that not only 
do the strategies evolve in time, but so does the environ- 
ment, and indeed many other factors that in turn affect 
back the outcome of the evolution of strategies. Some of 
these coevolutionary processes are of a finite duration, 
and thus on their own do not necessarily affect the out- 
come of evolutionary games but do this only indirectly 
due to the environment that they produce, while others 
are lasting, introducing dynamical alterations that affect 
the evolution of cooperation on a continuous basis. In 
the future, it should be of interest to further elaborate 
on the question whether coevolution itself may promote 
cooperation by introducing dynamical mechanisms, or 
if mainly the final outcome of a coevolutionary process, 
if it exists, is the one vital for the sustenance of coop- 
eration. Often, however, it is the interplay of both that 
facilitates the promotion of cooperation, as it was al- 
ready shown in some of the works. It should also be 
considered which coevolutionary processes end sooner 
or later, and which are those that at least in principle 
should last forever. For example, the growth of a city 
can be considered something that has a finite duration 
due to environmental constrains, while aging, on the 
other hand, is a natural ingredient of every living organ- 
ism, and as such it should only make sense to consider 
it evolving permanently. 

Although social dilemmas may emerge at differ- 
ent levels of human and animal interactions, their 
occurrence is by no means limited to these exam- 
ples. The applicability of the concept of evolution- 
ary games extends across the whole of social and nat- 
ural sciences, with examples ranging from the RNA 
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virus ( Turner and Chad 1 19991) . ATP-Producing Path 
ways (Pfeiffer et all l200lh and biochemical systems 



(Fric k and Schusteit 120031: iPfeiffer and Schusterl 12005 
Chettaoui et all 120071: ISchuster et all I2008h7 to traffic 



congestion dHelbing et all 120051; IPerd l2007ab and cli 
mate chan ge (Milinski et al. w - 
120061 iMilinski et all l20oi) 



, 2006: IPfeiffer and Nowak . 
to name but a few. In this 
sense revolutionary rules should be applied to evolu- 
tionary games in the broadest possible sense, with spe- 
cially adapted motivation fitting to the research avenue 
of the main evolutionary process. Moreover, while fo- 
cusing predominantly on resolving social dilemmas, co- 
evolutionary rules have thus far not been considered for 
many other game types, as for example the public goods 
game, the ultimatum game or the rock-scissors-paper 
game. These gaps should be interesting to fill as well, 
in particular when striving towards universal concepts 
underlying cooperation in the broadest possible sense. 
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